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Abstract. We study the polytopes of affine maps between two polytopes - the 
hom-polytopes. The hom-polytope functor has a left adjoint - tensor product poly- 
topes. The analogy with the category of vector spaces is limited, as we illustrate 
by a series of explicit examples exhibiting various extremal properties. The main 
challenge for hom-polytopes is to determine their vertices. A polytopal analogue of 
the rank-nullity theorem amounts to understanding how the vertex maps behave 
relative to their surjective and injective factors. This leads to interesting classes of 
surjective maps. In the last two sections we focus on two opposite extremal cases 
- when the source and target polytopes are both polygons and are either generic 
or regular. 

1. INTRODUCTION 

1.1. Motivation. The convex polytopes and their affine maps form a natural habi- 
tat for a major part of the contemporary combinatorics (combinatorial commutative 
algebra, toric algebraic geometry, tropical geometry, Ehrhart theory, linear and in- 
teger programming). The category of polytopes is also an object of study in its 
own right. The importance of such an approach is highlighted on the last pages 
of [13]. By analogy with algebraic structures and topological spaces, one could ask 
whether polytopes are also amenable to a unifying analysis, which would provide 
a general context for various important constructions and results. In the case of 
algebra and topology such a unifying machine is homology theory. Since convex 
polytopes are just one step away from the classical linear world, the question can be 
put in very concrete terms: what are the polytopal versions of Horn, 0, Ker, Coker, 
Ext? How do they fit into the current trends in polytope theory? Is there a more 
universal (algebraic?) mechanism for addressing concrete challenges in polytopes 
than, say, triangulations, analogous to the triangulations vs. homology dichotomy 
for topological spaces? 

A notable example that suggests that these are natural questions is the Billera- 
Sturmfels concept of fiber polytopes (3]. This construction is expected to be the 
right kernel object in the category of polytopes. But the analogy with ker is yet to 
be fully explained and, more importantly, pushed further to include still conjectural 
co-kernel objects in the same category. 
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In this paper we undertake the first step in the direction of categorial analysis of 
convex polytopes: we study the sets of affine maps between two given polytopes, the 
hom-polytopes. Curiously enough, apart from the motivation above, hom-polytopes 
show some relevance in quantum physics [9]. 



1.2. Results. That the hom-polytopes are in fact polytopes is a folklore fact; see 
Section [21 The full blown analogy with vector spaces is a symmetric closed monoidal 
structure on the category of polytopes over which the category itself is enriched 
(Corollary 13. 5p . In particular, there is a natural tensor product of polytopes, sat- 
isfying the usual conjunction with hom-polytopes. But, unlike the linear situation, 
the tensor product of polytopes exhibits interesting extremal properties (Example 
13. 7p . The material up to Corollary 13.51 is modeled on vector spaces and the argu- 
ments are mostly skipped. The summary is given for the sake of completeness. After 
posting the preprint on arXiv, we learned about Valby's undergraduate thesis [12] 
which gives a very detailed treatment of the same material. 

The description of the facets of hom-polytopes is very simple: a facet consists of 
the maps mapping a chosen vertex of the source into a chosen facet of the target. 
It is, therefore, the determination of vertices which amounts to understanding the 
geometric consequences of our categorial-polytopal endeavor. Very rarely can one 
hope for a full description of the vertices of hom-polytopes in terms of the source 
and target polytopes. In Section 0] we introduce several tractable classes of surjec- 
tive vertex maps in arbitrary dimension (deflations, face collapses). Their analysis 
(Theorems 14. II and 14.21) . in particular, yield complete description of the rank 1 ver- 
tex maps, i. e., the vertex maps whose images in the target polytope are segments 
(Corollary S3]). 

In Section [5] we present several examples of interesting vertex maps, making clear: 
(i) the limitation of the analogy between the categories of polytopes and vector 
spaces (vertex factorization of non-vertex maps, gaps in ranks), and (ii) the distinc- 
tion between the classes of surjective vertex maps, introduced in Section [U 

In Section E] we are able to show that in the hom-polytope between two generic 
polygons, all but a few vertices are simple (Theorem 16. ip . and we completely de- 
scribe the exceptions. (To this end we must first introduce an appropriate algebraic 
parametrization of the set of pairs of polygons.) The result is shown in two steps: 
(i) introducing combinatorial structures that reduce the problem to the claim that 
certain explicit multivariate polynomial determinants of 31 different types are non- 
degenerate, and then (ii) verifying the claim by effective methods, with use of the 
computer algebra system Macaulay 2. 

Regular polygons give rise to interesting arithmetic functions - the number of ver- 
tices of the hom-polytope Hom(P n , P m ) between regular n- and m-gons. In Section 
by explicit polygonal constructions, we obtain the explicit full lists of the vertices 
of Hom(P m ,P n ) when min(m, n) < 4. At the end of Section [7J we present computa- 
tional results, based on Polymake, for the number of vertices of Hom(P m ,P n ) with 
m, n < 8. 



HOM-POLYTOPES 



3 



1.3. Affine geometry. In this and next two subsections we fix terminology and 
collect several general facts. 

For the unexplained background material on convex polyhedral geometry the 
reader is referred to [U Ch.l] and [T31 Ch.1,2]. 

We will work exclusively in finite dimensional real vector spaces. An affine sub- 
space of a vector space is the sum of a linear subspace and a vector. For a vector 
space E and a subset X C E, the affine hull aff(X) is the minimal affine subspace 
of E containing X, and the linear hull lin(X) is the parallel translate of aff(X) 
containing 0: 

aff (X) = {\±xi + • ■ ■ + \ n x n | n G N, X\, . . . , x n G X, 

Ai, . . . , X n g R, Ai H h A n = 1}; 

lin(X) = {^i(xi — Xq) H h fi„(x n -x ) | n G N, x , x u . . . , x n G X, 

jtil, ...,fl n G R}. 

An affine map f : E — >■ E' between two vector spaces is the composition of a 
linear map E E' with a parallel translation E' — > A map / : E — > E' is affine 
if and only if it respects barycentric coordinates: 

(n \ n 

^XiXi ) = ^Xifixi) 
i=l J i=l 

for all n G N, x%, . . . , x n G E, and Ai, . . . , A n G R with Yli=x = 1- More generally, 
let A C E and A' C £" be affine subspaces. A map / : A — )■ A' is affine if it is 
a restriction of an affine map between the ambient vector spaces. The set of affine 
maps between E and E', denoted by aff (E, E'), is a vector space in a natural way. 
The set of affine maps A — > A', denoted by aff (A, A'), becomes an affine subspace of 
aff (E, E') upon choosing an affine projection ir : E — > A with it 2 = n and applying 
the embedding: 

&8{A,A') -> aft(E,E'), f^iofon, 

where i : A' — > E' is the identity embedding. The resulting affine structures on 
aff (A, A') for various ir are all isomorphic. As a result, there is a well-defined notion 
of convexity in aff (A, A'). This space satisfies dim aff (A, A') = dim A dim A'+dim A'. 

For a subset of an affine space X G A, its convex hull will be denoted by conv(A). 
For X C E, E a vector space, the conical hull of X will be denoted by cone(X): 

cone(X) = {1R + Xi H h R+x n | n G N, x x , . . . , x n G X} , 

where M + refers to the set of nonnegative reals. 

For a convex subset X C A, the relative interior of X in aff(X) C A will be 
denoted by int(X). The boundary of X is defined by dX — X \ int(X). 

For two convex sets X and Y, hom(X, Y) denotes the set of affine maps X — > Y 
and aff(X, Y) denotes the set of affine maps aff(X) — > aS(Y). The natural embed- 
ding hom(X, Y) > aff(X, Y) makes hom(X, Y) into a convex subset of aff(X, Y) 
that satisfies: 
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(1) aS(X,Y) = aff (hom(X, Y)) = hom(X, aff (Y)), 

(2) int(hom(X,r)) = hom(X, int(F)). 

1.4. Polytopes and cones. We only consider convex polytopes, i. e., our polytopes 
are the compact intersections of finitely many affine half-spaces, or equivalently the 
convex hulls of finitely many points. 

For a polytope P, the sets of its facets and vertices will be denoted by F(P) and 
vert(P), respectively. 

For two polytopes in their ambient vector spaces P C E and Q C E', their join 
is defined by 

join(P, Q) = conv{(a:, 0, 0), (0, 1, y) \ x G P, y G Q} C E © R © P'. 

Let ip and £q be the obvious embeddings of P and Q into join(P, Q). Every point 
z G join(P, Q) has a unique representation z = \ip(x) + (1 — \)iQ(y), X G [0, 1]. For 
two affine maps / : P — >■ P and g : Q — > R we have the affine map: 

join(P, Q) P, Ai P (x) + (1 - A)^(y) ^ A/(a;)+(l - X)g(y), 

A g [0,1], xeP, yeQ. 

It is uniquely determined by its restriction to Iuup and Im lq. 

By cones we will refer to pointed, convex, polyhedral cones: those that are ob- 
tained as the intersection of finitely many half-spaces and contain no lines. 

Further notation. The bipyramid over a polytope P in a vector space E is the 
polytope 

0(P) = conv((P, 0), c P + (0, 1), c P - (0, 1)) C E © R, 

where c p G P is the barycenter. 

The n-dimensional standard simplex, cube, and cross-polytope are defined by 

A n = conv(ei, ...,e n , e n+1 ), 

□ n = conv ( ^ $i e ii $i = ±l ) > 

^ i=i ' 
n = conv(±ei, . . . , ±e n ), 

where e, denotes the ith standard basis vector. 

For a natural number n > 3, the standard regular n-gon is 

P n = conv(l, Cn, C, ■ ■ ■ , Cr 1 ) C C = K © E, 
( n = cos(27r/n) + sin(27r/n)z. 

1.5. Categories. Our category theory terminology follows the classical source [S]: 
comma categories, limits and co-limits of diagrams, conjugated functors, and sym- 
metric monoidal categories. For the concept of enriched categories, we refer the 
reader to [7]. Even if this is the first time the reader encounters this terminology, 
the exposition is sufficiently self-explanatory to warrant skipping inclusion of the 
definitions. However, the interested reader can consult [12] . 
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Let Pol denote the category of polytopes and affine maps and Cones denote the 
category of cones and linear maps. In both categories, for objects A and B we will 
use the notation Hom(A, B) for the corresponding hom-sets. 

In Pol we have the following universal equalities: 

join(P,Q) = P]jQ = lim(P,Q), 
PxQ = Pl[Q = lim (P,Q). 

2. HOM-POLYTOPES 

The following proposition is well-known [131 §9-4] and is the basis for the special 
module in Polymake [TJ [5] for computing hom-polytopesE We used this software for 
the experiments presented in Section [7] below. The details are written up to ease 
the references in the following sections. 

Proposition 2.1. Let P,Q,R be polytopes. 

(1) Hom(P, Q) is a polytope in a.&(P,Q) with 

F(Hom(P,Q)) = {H(v,F) | v G vert(P), F G F(Q), 

H(v,F) = {fe Hom(P,Q) | f(v) G F}}, 

(2) dim Hom(P, Q) = dim P dim Q + dim Q . 

(3) Hom(P,Q x R) ^ Hom(P, Q) x Hom(P,P) ; 

(4) Hom(P, Q n R) = Hom(P, Q) n Hom(P, R) 

(5) Hom(join(P,Q),P) = Hom(P, R) x Hom(Q,P). 

Proof. (1) For every facet F C Q we fix a surjective affine map ^ : Q — )■ M + , 
vanishing only on P. 

Claim. The system of affine maps 

<p F , v : aff (P, Q) ^ R, f^{VF°f){v), F e ¥(Q), v E vert(P), 

defines the facets of Hom(P, Q). 

The equality 

Hom(P,Q) = f|{/ e aff (P, Q) | <p F , v {f) > 0} 

is straightforward. But for any vertex v G P and any facet F C Q, there is an affine 
map / : P — > Q such that f(v) G int(P) and /(u>) G int(Q) for each vertex w G P 
with w v. To obtain such map, take the composition of a parallel projection 
of P onto [0, 1], mapping v to and the rest of P onto (0, 1], with an embedding 
[0, 1] — > Q, mapping to int(P) and (0, 1] to mt(Q). Thus, 

fe<p F , v (p)~ l \ U Pg,™(0)-\ 

HQ) \ {F} 

w e vcrt(P) \ {v} 



Called mapping polytopes in Polymake. 
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which proves the claim. 

(2) In the notation introduced above, we have 

int(Horn(P,Q)) = f|{/ G aff(P,Q) | <p F , v {f) > 0}, 

F.v 

a nonempty bounded open subset of aff(P, Q) defined by linear inequalities, so 
Hom(P, Q) C aff (P, Q) is a full-dimensional polytope. 

The parts (3,4,5) follow from the universal equalities in Section rOl and the natural 
bijections of sets for any object a and any diagram T> in any category: 

Hom(a,limP) = limHom(a, T>), 

Hom(limP,a) = limHom(P,a); 

these bijections are affine maps in our polytopal setting. □ 
For cones we have the following analogous statment: 

Proposition 2.2. Let C\, C 2 , C 3 be cones. 

(1) Hom(Ci,C2) is a cone in Hom(lin(7i, linC^), whose facets are naturally in- 
dexed by the pairs (R, F) , R C C\ an extremal ray and F C C 2 a facet. 

(2) Hom(Ci, C 2 x C 3 ) = Hom(C7 1 , C 2 ) x Hom(Ci, C 3 ). 

(3) Hom(d x C 2 ,C 3 ) S Hom(C7i,C 3 ) x Hom(c7 2 , C 3 ). 

(4) Homfd, C 2 n C7 3 ) = Hom(C 1 , C 2 ) n Hom(C 1 , C 3 ). 

3. HOMOGENIZATION, DUALS, AND TENSOR PRODUCT 

To a polytope P in a vector space E we associate the homogenization cone: 
C(P) = cone(P, 1), where (P, 1) = {(x, 1) | ar G P} C P © R. 

A graded cone is a cone C together with an affine map if : C — > R + , satisfying 
the condition ^ _1 (0) = {0}. The map if is called a grading. All of our cones admit 
a grading [H Prop. 1.21]. For a graded cone (C,<p) we have the de-homogenization 
polytope <^ _1 (1), denoted by C[ij. 

For a full- dimensional polytope P with G int(P) and a full dimensional cone C 
in a vector space P, the corresponding dual objects in the dual space P are defined 
by 

P = {h G P I < 1 for all ar G P}, 
C = {h G P I /i(C) C M+} = Hom(L7,R + ). 

The duals are also full-dimensional in their ambient space E . 

For the general facts of dual polytopes and dual cones the reader is referred to [H 
§1.B] and [T31 §2.3]. (The latter uses 'polar' instead of 'dual'). 

Unless specified otherwise, for a polytope P in a vector space P, we consider the 

cone C(P) as a graded cone w.r.t. the grading C(P) C >■ E © R — P ^ > R . 
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For two polytopes P and Q, let ip and ip denote the gradings C(P), C(Q) — > K+, 
respectively. Then we have the grading on the product of the homogenization cones: 

C(P) x C(Q) -+ R+, (x, y) -+ ^x) + 4>(y), 

which results in the isomorphism of polytopes 

(C(P)xC(Q))[i]=jom(P,Q). 

Further, if dimP = dimP and G int(P) (i. e., (0, 1) G int(P, 1)), then C(P) 
will be viewed as a graded cone via the grading C(P) — > R+, h h-> /i((0, 1)). It is 
crucial that h((0, 1)) = iff h = 0. 

Definition 3.1. Let P and P' be vector spaces, C C P and C" C P' be cones, and 
P C P and Q C E' be polytopes. We define the tensor products as follows: 

C © C" = cone{(:r © y) \ x G C, y G C'} Cfi® P', 

P®Q = conv ({(x(g)y,x,y) | x G P, y G Q}) C (P © P') © P © P'. 

Proposition 3.2. Let C, Ci, C 2 , d fte cones and P be a polytope. 

(1) T/ie bilinear map C\ x d — )> Ci © d, (x, y) h-> x®y, solves the following 
universal problem: any bilinear map C\ x C 2 — > d passes through a unique 
linear map ip, 

d x C 2 =^> d © d ■ 




d 

Equivalently, Hom(d © C 2 , C 3 ) = Hom(Ci, Hom(C2, d)); z. e. 7 ©,Hom : 
Cones x Cones — > Cones form a pair of left and right adjoint functors. 

(2) Hom(Ci, C 2 T d © C 2 = Hom(d, Cf) for d and C 2 full- dimensional. 

(3) 

dim(Ci © C 2 ) = dim d dim d = dimHom(d,d), 

C © R+ = C, (d © C 2 ) © d = d © (C 2 © C 3 ), d © d ^ d © d, 
d © (d x d) = (d © d) x (d © d)- 

(4) The extremal rays of d © d ore £/ie tensor products of the extremal rays of 
d <™d d- 

(5) Hom(P, C) = Hom(C(P),C) ; where the Horn on the right refers to the set 
of affine maps from P to C . 

(6) If P is full- dimensional and G int(P) then: 

C(P ) C(P) as oraded cones, i. e., P C(P)f 1] , and 
Hom(P, [0, 1]) C(P ) n ((0, 1) - C(P )). 

Proof. (1-3) are straightforward analogues of the corresponding linear algebra facts. 

Notice. Actually, (2) holds true for general, not necessarily full dimensional, cones. 
This is so because C 00 = C for general cones. Here we have to restrict to the 
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full dimensional case because from the beginning we have restricted to the case of 
pointed cones, and the dual of a cone is pointed iff the cone is full dimensional. 

(4) Let Ri C Ci be extremal rays, Ui : Ci — > E + be linear maps with ur\0) = R h 
and (fii : Cj — >■ IR+ be gradings, i = 1,2. Then the linear map 

satisfies the condition 

(z/i ^^2 + ^1® ^) -1 (0) = Pi <8> P 2 . 

Conversely, let £ = Ylj=i x j ® be an element of an extremal ray R C Ci <E> C%, 
where Xj G C\ \ {0} and y.,- G C2 \ {0} for all j. Then Xj ®%jj^R for all j because 
Xj ^ for all j - the tensor product of two nonzero vectors is a nonzero vector. 
Consequently, there exist real numbers tj > with Xj ®yj = (tjXi) ®y\ for every j. 
In particular, R = WL + (xi <8> yi)- All we need to show is that x\ and y\ are extremal 
generators of C\ and C 2 , respectively. Without loss of generality we can assume 
that x\ is not an extremal generator of C\. There is a segment [u, v] G C% such that 
x\ G (u,v) and ^ aff([w, u]). But then the subset [u ® yi, v ® yi] C Ci <g) C 2 is a 
nondegenerate segment and ^ aff (u g) yi, f <8> yi) - one uses the same fact on the 
tensor product of nonzero vectors. This contradicts the assumption that X\®yi G R. 

(5) We have the mutually inverse linear maps: 

a : Hom(P, C) ->■ Hom(C(P), C), a(h)((x, z)) = h{z~ x x), (x, z) G C(P) \ {0}, 
P : Hom(C7(P),C) -> Hom(P,C), /9(/i)(x) = 1)), x G P. 

(6) We have the following mutually inverse affine maps: 

7 . P ^ c^P)^, 7 (/ l )((x, ^)) = z - z ■ h(z~ l x), (x, z) G C(P) \ {0}, 

5 :C(P) {1] ^ P , 6(h)(x) = l-h({x,l)), xeP. 

That 7 and 5 in fact evaluate in the right objects and have all necessary properties 
is straightorward. 

For the second isomorphism, we note that [0, 1] = R+. Pi (1 — R+) and so (5), the 
previous isomorphism, and Proposition 12.2( 4) apply. □ 

Remark 3.3. (1) Caution is needed in extending linear algebra facts to cones. For 
instance, the canonical map C^(g)C| — > (Ci®^) is in general not an isomorphism: 
by Proposition 13.2( 4). the extremal rays of the first cone naturally correspond to the 
pairs of facets Pi C C\ and P 2 C C 2 , while those of the second one are in bijective 
correspondence with the facets of C\ <8> C 2 . But there is no natural correspondence 
between the two sets. In fact, they can easily be non-bijective - a combined effect 
of Proposition 12.1( 1) and the upcoming Proposition 13.4( 1). 

(2) As a consequence of (6), for two isomorphic full dimensional polytopes P\ = P 2 
with G int(Pi) flint (P 2 ), although Pf and P| are in general not isomorphic in Pol, 
we nonetheless have the isomorphism 

c{pf) n ((0, 1) - c{pf)) - C(P|) n ((o, 1) - c(p|)). 
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In particular, we recover the standard fact that, when a full-dimensional polytope 
moves around the origin so that the origin stays in the interior of the polytope, the 
resulting dual polytopes are all projectively equivalent. 

For polytopes P, Q, R, a map P x Q — > R is called bi-affine if, upon fixing one 
component, it is affine w.r.t. the other component. 

Proposition 3.4. Let P, Q, R be polytopes. 

(1) C(P ®Q) = C(P) g C{Q), or, equivalents, 

vert(P g Q) — {(v g w, v, w) \ v G vert(P), w G vert(Q)}. 

(2) The bi-affine map P 'xQ — > P®Q, (x,y) h-> (x®y,x,y), solves the following 
universal problem: any bi-affine map P x Q — y R passes through a unique 
affine map if, 

P xQ^^>P®Q . 

/^C V 

R 

Equivalently, Hom(P<g>Q, R) = Hom(P, Hom(Q, R)), i.e., g, Horn : Pol x Pol ->■ 
Pol form a pair of left and right adjoint functors. 

(3) 

dim(P g Q) — dim P dim Q + dim P + dim Q, 

P g {*} S P, (P g Q) g P S P g (Q g P), P®Q^Q®P. 
P g join(Q, P) = join(P g Q, P g P), 
vert(P g Q) = {(f g tu, t> , to) | t> G vert(P), u> G vert(Q)}. 
Proof. (1) We have 

C(P) g C(Q) =cone{(x g y) | x G C(P), y G C(Q)} = 
cone{(x ® y) | x G (P, 1), y G (Q, 1)} = 
cone{(w <S)v,u,v,l) \ u G P, «6Q} = C(P g Q). 

(2) A bi-affine map ip : P x Q > R gives rise to the following bilinear map: 

C(P) x C(Q) => C(P) , 

(0, a?), 0, y)) >->■ (a^/ • V (z~V y -1 ^) , ary) , x, y > 0. 
By Proposition 13.2( 1). we have the linear map: 
C(P)gC(Q) ->C(P), 

(u, x) g (v, y) i — y {xy ■ ip (x~ u, y~ v ) , xy) , x, y > 0, 
which shows that the composite affine map 

P g Q 9* (P g Q, l) C(P g Q) C(P) g C(Q) ->• C(P), 
evaluates in (P, 1). The uniqueness is straightforward. 

(3) This follows from (1) and the corresponding parts of Proposition 13.21 □ 



10 T. BOGART, M. CONTOIS, AND J. GUBELADZE 

For polytopes P, Q, R, the pairing 

Hom(P, Q) x Hom(Q, R) Hom(P, R), (f, g)^gof, 

is clearly bi-affine. The pentagon and hexagon coherence conditions of the bifunctor 
(g) : Pol x Pol — » Pol are inherited from the corresponding conditions of the tensor 
product of vector spaces. So Proposition 13.41 has the following consequence. 

Corollary 3.5. (Pol,®) is a symmetric closed monoidal category w.r.t. which Pol 
is self- enriched. 

As a more computational application, we have the following. 

Corollary 3.6. Let P be a polytope and n and m be natural numbers. Then: 

(1) Hom(A n ,P) = P n+1 and A n <g> P = pi°™ n+1 - the (n + l)-fold iteration of 
join, applied to P. 

(2) Hom(P, □„) = (}(P ) n for P centrally symmetric w.r.t. 0. In particular, 
Hom(D m ,D n ) = (0 m+ i) n - 

(3) Hom(D m , On) = Hom(D n _ 1 , m +i)- 

Proof. (1) Any map vert(A n ) — > P extends uniquely to an affine map A n — > P. 

For the second isomorphism, one uses Proposition 13.4( 3). the third isomorphism, 
and the fact that A n = (*)j° in " +1 . 

(2) This follows from Propositions 12. 1( 3) and 13.2( 6). the 2nd isomorphism, and the 
fact that P is also symmetric w.r.t. the origin. (See Remark 13.3( 2).) 

(3) One applies the conjunction £g> H Horn twice. □ 

We have not been able to find rank > 2 elements of vert(Hom(D m , A n )). More 
generally, one would wish to have Corollary 13 . 6 1 completed by an explicit description 
of the hom-polytopes between the general regular polytopes. Dimensions 3 and 4 
seem more challenging than dim > 5. Currently, even the 2-dimensional case, or 
just a satisfactory description of vert(Hom(P n , P m )), is out of reach. Partial results 
in the latter direction are presented in Section [71 

In the context of extremal maps from regular polytopes, it is interesting to remark 
that the octahedron O3 admits an affine embedding into any simple 3-dimensional 
polytope P so that the vertices of O3 map to the boundary dP [2]. 

We conclude the section with one application of the tensor product, yielding point 
configurations with interesting extremal properties. 

Example 3.7. Let P and Q be polytopes in a vector space E. 

(1) For a vertex v G P and an edge [wi,u>2] C Q, the segment 

[(v <g> w h v, wx), (v <g> w 2 , v, w 2 )\ C E® 2 © P® 2 
is easily seen to be an edge of P g) Q. 

(2) The tensor square P® 2 = P <8> P admits less obvious edges: 

[(v <g> v, v , v ), (w <8> w, w, w)} C P® 2 , v, w £ vert(P), v 7^ w. 
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To show this, assume v, w G vert(P), v ^ w and let <p v ,<p w : C{P) — > E + be linear 
maps, vanishing on exactly cone(f') and cone(w/), respectively, where v' = (v, 1) and 
w' — {w, 1). Then the linear map 



(the isomorphism on the left is from Proposition 13.4( 1)). vanishes on exactly the 
2-face 

cone(V <g> v', w' ® u/) C C(P® 2 ). 

As a consequence of the two types of edges, if P has m vertices and n edges, then 
the tensor square P® 2 has at least 2mn + m ^~ 1 ^ edges. This counting is, however, 
far from the complete list; an example is (A n )® 2 = A n 2 +2n . 

(3) It follows from (2) that the following polytope is neighborly: 

conv {{v <S>v,v,v) \ ve vert(P)) C P® 2 © P® 2 , 

i. e., any two vertices are joined by an edge. (In [T3J Ch.O] the property is called 
2-neighborly.) 

(4) A similar argument implies that for the unit Euclidean ball B d C M d and any 
two points v,w G S^ 1 = dB d , the segment 

[{v <g> v, v, v),(w® w, w, w)] C {B d )® 2 (= conv((x ®y,x,y) \ x,y G B d )) 

is an extremal subset (jH p. 10]). 

(5) Similarly, the map 



has the following extremal property: for any system of points t> i, . . . , v n G S^ -1 , their 
^d_i-images are in convex and neighborly position. This observation is, in a sense, 
weaker than (4) because conv^m^-i) ^ {Bd)® ■ (For instance, {v ® w, v, w) G 
(B d )® \ conv(Im^_i) for v,w G if> d-1 , u 7^ w.) On the other hand, it yields the 
interesting embedding 

ip : S 1 — > 1R 4 , (cost, sin t) i-)- (cost, sin t, cos 2t, sin 2t), 

which maps any number of points on the unit circle into a convex neighborly point 
configuration in M 4 . In fact, ip is obtained from ipi by the following series of affine 
transformations of 1R 8 

(cos t, sin t) 1 — y (cos 2 1, sin 2 1, cos t sin t, cos t sin t, cos t, sin t, cos t, sin t) 1— >• 
(cos £, sin £, cos 2 £, sin 2 1, cos £ sin £) 1— > (cos £, sin £, cos 2 £, 1 — cos 2 £, sin 2£) \— > 
(cos £ , sin £, cos 2 £, sin 2£) 1— >■ (cos £, sin £, cos 2£, sin 2£) , 

implying an affine isomorphism conv(Im^ 1 ) = conv(Im?/>). 
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4. Vertex factorizations 

Let A and B be convex sets. A family of maps (ft)(_ l ^ C Hom(A, B) is called 
an affine 1-family if the map 

(-1,1) ^Hom(A,P), t^f t , 

is injective and affine. 

A map / : P — > Q in Pol is called a vertex map, or just a vertex, if / G 
vert(Hom(P,Q)). 

We will need the following obvious (affine) perturbation criteria for vertices and 
interior points. Let / : P — > Q be an affine map between two polytopes. Then: 
(pc-J / is not a vertex if and only if there is an affine 1-family C 

Hom(P,Q),with/ = /. 
(pc 2 ) / G int(Hom(P, Q)) if and only if for any affine 1-family (/t)^_ 1 ^ C aff(P, Q) 

with fo — f there exists a real number £ > such that (/ t ) ^ C Hom(P, Q) 

Every map / in Pol factors, uniquely up to the obvious equivalence, into a sur- 
jective and an injective map in Pol: / = /i n j o / sur j. 

A map / : P — > Q is called a deflation if it satisfies the conditions: 

(i) / is surjective, 

(ii) / is a vertex, 

(iii) for any vertex v G P either f(v) G vert(Q) or f{y) G int(Q). 

Simple examples of deflations are provided by the parallel projections □„ — > D m 
along the subspace R+e m+ i + • • • + M+e„ C K n , where we assume m < n. 
The ranA; of a map / in Pol is defined by rank / = dimlm(/). 

Theorem 4.1. Let f : P Q be a map in Pol. 

(1) If f is a vertex then so are / in j and / sur j. 

(2) 7/vert(/(P)) C vert(Q) and / sur j is a deflation then f is also a vertex map. 

(3) There are examples of non-vertex maps f with f\ n j a vertex map and / sur j a 
deflation. In particular, the converse to the implication in (1) is not true. 

Proof. We can assume / sur j : P — > f{P). 

(1) (Using (pc 1 ).) If /i n j : /(P) — >■ Q is not a vertex then there exists an affine 1- 
family (V't)(-i,i) C Hom(/(P),<5) with = / in j. Then the resulting affine 1-family 
(V't ° /surj) ( _ 1;1) C Hom(P, Q) has ° /surj = /, a contradiction. 

If /surj is not vertex then there is an affine 1-family (</?t)(-i,i) C Hom(P, /(P)) 
with <y? = / sur j. So we get the affine 1-family (/ in j o </ ? t)(_ 11 ) c Hom(P, Q) with 
/inj ° — /? again a contradiction. 

(2) Observe that, in view of (pc-J, the condition vert(/(P)) C vert(Q) already 
implies that f\ n j is a vertex map. But we do not use this explicitly. 

Assume to the contrary that / is not a vertex. By (pc x ), there is an affine 1-family 
(/t)(-i,i) cHom(P,Q) with /„ = /. 

We will show that the maps 

(p v :(-l,l)->Q, t^f t (v), 
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are constant for all v G vert(P). Since this implies that the map 

(-l,l)^Hom(P,Q), t^f u 

is constant, we have the desired contradiction. 
The maps 

(p v : (-1, 1) ->■ Q, t^f t (v), 

are constant for the vertices v G vert(P) with f(v) G vert(Q). 
Assume v\, . . . , v k are the vertices of P for which the maps 

<f Vl ,...,if Vk : (-1,1) -*•(?, tt->f t (v k ), 

are not constant. By the hypothesis, f(vi), . . . , f(v k ) G int(/(P)). 

Fix a (dimQ — rank /) -dimensional affine subspace H C aff(Q) that is parallel to 
none of the intervals 

U ft(vi)cQ, i = l,..., A;, 

te(-i,i) 

and consider the parallel projection it : Q — > aff(Im(/)) along H. We get an affine 
family (it o / t )(_i,i) C= aff (P, Im(/)) with vr o / = / surj . By (pc 2 ), there exists a 
real number e > which defines the affine family (it o f t )/_ E ^ c Hom(P, Im(/)), 
contradicting the assumption that / sur j is vertex. 

(3) Examples of non-vertex maps / for which f m j is a vertex and / sur j is a deflation 
will be presented in Example 15.11 □ 

Let P be a polytope and < r < dim P. Consider a family T = {G\, . . . , Gk}, 
satisfying the conditions: 

(i) Gi C P is a face and dim Gi > 0, i — 1, . . . , k, 

(ii) codim(lin(Gi) H h lin(G fc )) = r, 

(iii) (Gj + Mn(G 1 ) + ■■■ + lin(G fc )) D P = G<, i = 1, . . . , 

(iv) the family of faces {Gi, . . . , Gk} is maximal w.r.t inclusion among the families 
satisfying the first three conditions. 

The linear hulls above are taken in the ambient vector space and the codimension 
is understood relative to lin(P). 

To T we associate a surjective affine map of rank r from P as follows. We can 
assume G P. Consider a linear map / : lin(P) — > E with dimP = r and 
ker(/) = lin(Cri) H — ■ + lin(Gfc). For the restriction f\p we will use the notation fr. 

A map P — > Q in Pol is called a face-collapse of rank r if it is of the type fr up 
to isomorphism in the comma category P 4- Pol. 

The following example shows that neither of the conditions (iii) and (iv) above 
can be dropped in the definition of face-collapses. Let Pq be a regular hexagon with 
vertices i>i, . . . , t> 6 . Then the pair of facets T = {[vi, v 2 ] x [0, 1], [t> 4 , v 5 ] x [0, 1]} of the 
unit prism P 6 x [0, 1] satisfies (i-iv). On the one hand, if one drops (iii), then T ceases 
to be maximal. On the other hand, if one drops (iv), then T' = {[f 1,1*2] x [0,1]} 
satisfies (i,ii,iii), but one cannot collapse the facet [^1,^2] x [0, 1] into a point by an 
affine map without also collapsing the facet [^4,^5] x [0,1]: 
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Figure 1. A polytope illustrating the definition of face-collapse 



Theorem 4.2. We have the following proper containments: 

Deflations ^ Face- Collapses ^ Surjective Vertex Maps. 

Proof. First we show that face-collapses are vertex maps. 

Let P G Pol and T = {G\, . . . ,Gk} be a system of faces of P satisfying the 
condition (i-iv) above. Without loss of generality we can assume G P. Denote 
if = lm(Gi) + ... + lin(G fc ). 

First we observe that /(Gj) G vert(/(P)) for every i — 1, . . . , k. In fact, /(Gj) C 
df(P) by (iii) and dim /(Gj) = 0. But then, if /(Gj) G int(7) for some positive 
dimensional face 7 C f{P), we have int(Gj) C int(/ r T 1 (7)) and /f" 1 ^) is a face of 
P with Gj ^ f r 1 (7). This is impossible because Gj C P is a face. 

Suppose /r is not a vertex. By (pc x ), there exists an affine 1-family (/t)(-i,i) C 
Hom(P,/(P)) with f = f r . The condition f{Gi) G vert(/(P)) forces / t | Gj = 
f\d for all £ G (—1, 1) and i = 1, . . . , k. In other words, the 1-family (ft)(-i,i) is 
constant on every Gj. Let {lUfc+i, . . . , w k+i } = vert(/(P)) \ {/(Gi), . . . , f(G k )}. Fix 
Ufc+i, • • • , ffc+/ G vert(P) with f(vj) = Wj, j = k + 1, . . . , k + I. (The Vj are uniquely 
determined, but we do not need this.) Using again (pcjj, the 1-family (ft) (-1,1) is 
constant on the Vj. So (ft)(-i,i) is constant on conv(Gi, . . . , G k , ffc+i, • • • , ffc+j)- But, 
by (ii), the latter is a full dimensional subpolytope of P and that forces our 1-family 
to be constant on P - a contradiction. 

Next we show that deflations are face-collapses. 

Consider a rank r deflation / G vert (Hom(P, f(P)). We can assume that G P 
and that /(0) = 0; i.e., / is the restriction of a (unique) linear map h : lin(P) — > 
lin(/(P)). Let w 1: ... : w k G f(P) be the vertices with dim f~ 1 (w i ) > for all 
i — 1, . . . , k. It is enough to show the following 

Claim. The faces G« = f~ 1 (w i ) C P, i = 1, form a system satisfying the 

conditions (i-iv) with ker(/i) = lin(Gi) + h lin(Gfc). 
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The conditions (i,iii,iv) are straightforward. Since lin(Gi) + - ■ ■ + lin(G/ c ) C ker(h), 
we only need to verify (ii). 

Assume to the contrary codim ( lin(Gi)+- • •+lin(Gfc)) > r. Let {wk+i, ■ ■ ■ , w k+ i} = 
vert(/(P)) \ {wi, . . . , w k }. There are (uniquely determined) vertices Vk+i, ■ ■ ■ , v k+ i G 
P with f(vj) = Wj for j = k + 1, . . . ,k + I. Our assumption implies 

dimconv(Gi, . . .,G k ,v k+1 , . . .,v k+l ) < dim P. 

Without loss of generality we can additionally assume 

G conv(Gi, ...,G k , v k+h v k+ i). 

Pick a basis P C lin(P), restricting to a basis P G lin(Gi, . . . , G k , v k+ i, . . . , ffc+j), 
and a basis P' C lin(/(P)). Let M be the matrix of h with respect to the bases 
P and B' so that the first #(Po) rows correspond to the elements of B . Consider 
the affine 1-family (/if)(_i 5 i) G Hom(lin(P), lin(/(P))), where the matrix of h t in 
the bases B and B' is obtained from M by adding (t, . . . , t) G 1R*' 8 ' to each of 
the last #(P \ Po) rows. Then, because / is a deflation, there exists < e < 1 
such that h t (P) C /(P) for all — e < t < e and this produces an affine 1-family in 
Hom(P, /(P)), containing / - a contradiction by (pcx). 

Examples 15. 2l and l5.3l in the next section will show that both inclusions in Theorem 
14.21 are proper inclusions. □ 

Corollary 4.3. (1) A rank 1 surjective map is a vertex if and only if it is a 
face-collapse if and only if it is a deflation. 

(2) Every system F of faces of P satisfying (i-iv) and rank(/r) = 1 has at most 
two elements. If P is centrally symmetric then such systems T are exactly 
the pairs of opposite facets of P. 

(3) A rank one map f is a vertex iff / sur j and /i n j are vertices. 

(4) For a polygon P with I edges, of which m pairs are parallel, and a polytope Q 
with n vertices, the number of rank 1 vertex maps P — >■ Q is (/ — m)n(n — 1) . 

Proof. (1) holds true because of the first inclusion in Theorem 14.21 and the fact that 
the boundary of a segment is just the vertices of the segment . 

(2) If {u,v} = vert/ r (P) and dim f~ l (u), dim" 1 (v) > then T = {f' 1 ^),/- 1 ^)}; 
otherwise F consists of a single facet of P. The conclusion for centrally symmetric 
polytopes is obvious. 

(3) From (pc^), the injective vertex maps from a segment to Q are the maps that 
send the two vertices of the segment to distinct vertices of Q. So the claim follows 
from (1) and Theorem 14.1( 1.2). 



(4) In view of (1) and (2), (I — m) is the number of surjective vertex maps from P 
to a segment. The count follows from (3) and the observation above on injective 
vertex maps from a segment. □ 
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FIGURE 2. The construction of Example 15.11 with n = 3 

5. Examples of vertex maps 

In this section we present various constructions of vertex maps exhibiting inter- 
esting phenomena, some of them mentioned in the previous sections. 

Example 5.1 (Vertex factorization of non-vertex maps). For a map / : P — > Q in 
Pol, if 1 + dimP affinely independent vertices of P map to vertices of Q then / is a 
vertex. In fact, if there were an affine 1-family (/t)(-i,i) C Hom(P, Q) with fo = f, 
then the family would be constant on the distinguished vertices of P. The affine 
hull of these vertices is all of aff(P). Consequently, (ft) must be constant on P 
- a contradiction. This simple observation, together with Theorem 14.1( 2). suggests 
that to look for non-vertex maps / : P — > Q with / sur j and /i n j both vertices, we 
should consider maps that keep the vertices of Im/ away from vert(Q). We first 
give a general construction and then construct examples of vertex factorizations of 
non-vertex maps. 

Let Q C 1R 3 be a 3-polytope and H C M 3 be an affine plane such that: (i) the 
intersection P = Q fl H is a polygon with 2n > 6 vertices, and (ii) there exist 
Wi, . . . ,W2n € vert(Q) such that every open interval (wi,Wi + i) contains exactly one 
vertex Vi G vert(P), the indexing being mod 2n. Observe that the segments [wi, iUt+i] 
are necessarily edges of Q. 

Claim. The identity embedding t : P — > Q is a vertex of Hom(P, Q). 

The two groups of vertices Wi, w 3 , . . . , w 2n -i and w 2 , w^, . . . , w 2n are separated by 
H . To see that i is a vertex, we apply the following sliding argument. Assume 
t $l vert (Horn (P, Q)). Pick a vertex of a £ Hom(P, Q), which belongs to the same 
minimal face of Hom(P, Q) as i. By sliding i along the segment [t, a] one gets a 
1-parameter family (k)[o,i] C Hom(P, Q) with l q = t and i\ = o such that, for 
every t £ [0,1], the image of it is a polygon, isomorphic to P and with vertices 
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Figure 3. The construction of Example 15. II with P n regular, n = 4 

v t i G [wi, Wi+i]. In fact, the 'vertex of h(P) <-> edge of Q J incidence table remains 
constant in the process of sliding for t G [0,1). So dim(Imt t ) = 2 for t G [0,1). 
But dimlmcr = 2 as well because no affine line can intersect all segments [u^Wj+i] 
simultaneously - this is where we use the inequality n > 60 In particular, Im^) -P 
for all t G [0,1]. Since none of the mentioned incidences is lost for t = 1 and 
a is a vertex, a belongs to more facets of Hom(P, Q) than l. So the condition 
vn G [wi, w i+ i] for i = 1, . . . , 2n forces v\j = Wj or v\j = Wj + % for some j. But then 
one easily deduces wi, . . . , w 2n G Imu, forcing dim(Ima) = 3. This contradicts the 
fact that Im a = P, proving the claim. 

Consider the special case of the construction above, using the regular n-gon P n 
with n even: 

Q = conv(P n , P n ) C C © E, with 
P n = (P n , -1) C C © R, 
P n = {r]P n , 1) C C © R, 
rj = cos(7r/ra) + sin(7r/n)i. 

For every h G (—1, 1), the polygon P^ = Q n (C, h), is a centrally symmetric 2n-gon 
and the identity embedding 6 ft : P ft — > Q is a vertex of Hom(P ft , Q). We can choose 
surjective maps ph '■ On Ph, h G (—1, 1) so that (i) for every h, vertices map to 
vertices and (ii) the maps ph, viewed as elements of aff(IR ra ,C © R), continuously 
depend on h. Because each p h maps vertices to vertices, we have ph G vert(0n 5 P/i) 
for every h. Moreover, ph is a deflation for every h. 

It is important that the assignment h h- > ph is not just continuous, but even an 
affine map (-1, 1) -> aff(M n , C © E). 



For n = 2 the construction does not go through; an example is a tetrahedron and a plane, 
intersecting the tetrahedron in a parallelogram - the identity embedding of the parallelogram into 
the tetrahedron is not a vertex. 
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Figure 4. A face-collapse that is not a deflation 

Summarizing, for every h G (—1,1), the composite map fh = ('h Ph has the 
surjective factor (/h) sur j = Ph a deflation and the injective factor (fh)mj — a 
vertex map. Yet, 

/„ £ vert(Hom(0n,<5))- 
For simplicity of notation we can restrict to the case h — 0, and then f fits into the 
affine 1-family (f h )he(~i,i) C Hom(0 n ,<5)- 

From the construction above one can derive new examples of vertex factorizations 
of non-vertex maps as follows. Observe that, for every h G (— 1, 1), there exists a 
deflation : □„ — > Ph- So the composite maps gn = Lh ° T~h do not belong to 
vert(Hom(D n , Q)), yet (ghjsuij is a deflation and (gh)m] is a vertex map. 

Example 5.2 (Face- collapses that are not deflations). Not all face-collapses are 
deflations. Here is an example. Let {ei,e 2 ,e 3 } C M 3 be the standard basis and 
consider the parallel projection in the direction of —e^. 

n : conv(0, 2ei, 2e 2 , e 3 , e\ + e 3 , e 2 + e 3 ) ->■ conv(0, 2ei, 2e 2 ) 

We have tt = fr for T = {[0, e 3 ]}. However, tt is not a deflation because 7r(ei + e 3 ) 
and 7r(e 2 + e 3 ) are neither vertices not interior points of the target polytope. 

Example 5.3 (Not all surjective vertex maps are face-collapses). By a slight mod- 
ification of the map above, we get an example of a surjective vertex map which is 
not a face-collapse. Consider the parallel projection in the direction — e 3 : 

p : conv(0, 2e l , 2e 2 , e 1 + e 3 , e 2 + e 3 ) ->■ conv(0, 2e x , 2e 2 ) 

This projection cannot be a face-collapse because none of the positive dimensional 
faces of the source polytope is collapsed into a point. To show that p is a vertex, 
assume there is an affine 1-family (p t ) (-1,1) with po = p. Then it must be constant on 
the vertices of conv(0, 2e±, 2e 2 ). But it must also be constant on ei+e 3 and e 2 +e 3 . In 
fact, if the family is not constant there then pt(ei + e 3 ) and pt(e 2 + e 3 ) must trace out 



HOM-POLYTOPES 



19 




Figure 5. A surjective vertex map that is not a face-collapse 

parallel intervals as t varies over (—1,1) (one uses barycentric coordinates). But, on 
the other hand, these trajectories must be confined to the non-parallel edges [0, 2ej] 
and [0, 2e 2 ] of the target polytope. So the family (pt) (-1,1) is constant on all vertices 
of the source polytope, forcing the family to be constant - a contradiction. 

Example 5.4 (Many incident facets) . As we know, dim(Hom(P n , P n )) = 6 for the 
regular n-gon, n G N. By Proposition 12.1( 1). any automorphism P n — > P n sits on 2n 
facets of Hom(P n , P n ). These 2n facets can be split into two groups of n facets, each 
defining an edge of Hom(P n ,P n ). For the identity map, these groups are specified 
as follows. 



Fx = {H(i(£)), [C*,C* +1 ]) C Hom(P n ,P n ) | k E Z} , 
F 2 = {H(i{£)), [(t\(n\) C Hom(P n ,P n ) | k € Z} , 
notation as in Proposition 12.1( 1). The edges are, correspondingly, 

B, = f> = : P„ ^ P,„ C„ l ^ (i - C„* + Q + C„' +1 } (s[ t , and 

*=f> = ^g-Ocr + (i + 04 6i _ M . 

We leave to the reader to check that Ei and E 2 are in fact the edges, joining the 
identity map with the rotations by 27r/n and — 2ir/n, respectively. 

Example 5.5 (Gaps in ranks). For two polytopes P and Q, the range of possible 
ranks r of the vertices of Hom(P, Q) is 

< r < min(dimP, dimQ). 

For any polytopes P and Q, the polytope Hom(P, Q) always has rank vertices. If 
dim P, dim Q > then there are also rank 1 vertices, explicitly described in Corollary 
14.31 However, there may be gaps in the range. 

Claim. For any natural number n there are no rank 2 vertices in Hom(0n, A 2 ). 
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Figure 6. Two edges of Hom(P n ,P n ) incident to the identity map, 
n = 6 

First, for every polytope P and an element / G vert (On, P), one has 

vert(Im/) C vert(P n (-P (c) )), 

where c is the center of Im/ and — p( c ) is the symmetric image of P w.r.t. c. In 
fact, because Im/ is centrally symmetric, we have Im/ C P D (— P^). But if 
there is a vertex u G Im/ not in vert(P fl (— P^)), then there is an open interval 
/ C Im/, containing t>. Then, by sliding t> along / (and — along — 1^ in the 
opposite direction), one can define an affine 1-parameter (/ t ) C Hom(0 n ?-P) 
with fo = f. Such is not possible in view of (pc x ). 

Returning to the case P = A2, assume to the contrary that / is a rank 2 vertex of 
Hom(0«, A2). Let c be the center of Im/. There are two cases: (i) A2 fl (— A^) is 
a parallelogram, or (ii) A2 fl (— Aj ) is a centrally symmetric hexagon. In the first 
case, there is a vertex x G A 2 such that — x^ c ' is in the interior of the edge E C A 2 , 
opposite to x. So sliding c along a small open interval c G J, parallel to P, produces 
an affine family of parallelograms (A 2 n (-A 2 c) ))/ in A 2 . The latter can be used to 
define an affine 1-parameter family (ft) (-1,1) C Hom(0n, A 2 ) with f = f. In the 
second case, we use the similar sliding procedure, except now the small open interval 
/ C A 2 , containing c, is not constrained to have any particular direction - we can 
always define an affine family of centrally symmetric hexagons (A 2 fl (— Aj ))/. In 
either case we get a contradiction by (pc-J. 

As a side observation, the right picture in Figure [7] gives rise to yet another 
example of a deflation, followed by an injective vertex map, such that the composite 
is not a vertex map. In fact, the deflation is a surjective affine map O3 — > A2 fl A^ 
and the injective vertex map is the embedding A 2 fl Ac — > A 2 . One needs to apply 
a variant of the sliding argument in Example 15.11 to show that the latter belongs to 
vert (A2 fl Ac , A2). The crucial point is that no perturbation of the hexagon in 
A2 can keep both the isomorphism class of the hexagon and the 'hexagon vertex -H- 
triangle edge' incidence table constant. 

6. Generic pairs of polygons 

The goal of this section is to understand some of the structure of the hom-polytope 
of a generic pair of polygons (P,Q). The main result, Theorem 16. 1[ is that such a 
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FIGURE 7. The sliding argument in Example 15.51 



hom-polytope is 'almost simple': apart from the short list of vertex maps of rank 
zero and one, every vertex of Hom(P, Q) is simple. 

We begin by constructing spaces of polytopes that will allow us to precisely state 
this result. Fix dimensions d and e and integers m>d+l,n>e + l. To every 
real d x m matrix M we associate the polytope P(M) C ]R d given as the convex 
hull of the columns of M . Let Rd, m ^ ]R dxm be the set of matrices M for which the 
columns are in convex position and affmely span IR m ; that is, for which P(M) is a 
<i-polytope with m vertices. Similarly, to every real n x e matrix M' we associate 
the polyhedron Q(M') C ]R e given by the system of inequalities M'x < 1, where 
1 e M. n denotes the vector of all ones. Let R' e n be the set of matrices M' for which 
Q(M') is an e-polytope with n facets. 

Note that the conditions defining Rd, m and R' e n are stable under small pertur- 
bation; that is, Rd, m and R' en are open subsets of M. dxm and M nxe , respectively. 
Also, for generic M and M', P(M) is simplicial and Q(M') is simple. Furthermore, 
Rd, m C R dxm and R' e n C IR nxe are semi- algebraic subsets (i. e., defined by algebraic 
equalities and (strict) inequalities). In fact, the convex ci-polytopes with m- vertices 
give rise to only finitely many combinatorial types and each type is represented by a 
semi-algebraic subset of IR dxm , consisting of the matrices whose certain dx d minors 
vanish, certain d x d minors are positive, and certain d x d minors are negative. 
Therefore, Rd, m is the union of finitely many semi-algebraic sets and, as such, is 
itself semi- algebraic. A similar argument applies to R' e n . 

Clearly, R2, m and R' 2 n are the realization spaces of single combinatorial types. 
Moreover, the realization space of any 3-dimensional combinatorial type is a clas- 
sically a smooth ball; however, starting from d — e = 4, the realization space of a 
combinatorial type can be arbitrarily complicated; see [TU] . 

It is in the context above that we consider the hom-polytopes Hom(P(M), Q(M')) 
for a generic pair (M, M'). 

We now focus on the case of pairs of polygons: d = e = 2. Every m-gon in 
the plane is associated to a matrix in i?2,m that is unique up to permuting its 
columns. Every n-gon in the plane that contains the origin in its interior is associated 
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to a matrix in R' 2n that is unique up to permuting its rows. So i?2,m x a 
full- dimensional semi-algebraic open subset of the Euclidean space IR 2xm x M nx2 , 
effectively represents the space of pairs of polygons. 

One the one hand, we have dim(i?2, m x R'2 n ) = ^ m + ^n. On the other hand, for 
every pair (M , M') G dim(_R 2 ,m x -R^n); the space of all small perturbations of the 
facets of Hom(P(M), Q(M')), keeping the facet-normals invariant, has dimension 
ran (by Proposition 12 . 1 ( 1 ) ) . Yet, we have 

Theorem 6.1. There is a dense open subset U m ^ n of R2, m x P^n such that if 
(M, M') G U m ,n, then every vertex map f : P(M) — > Q(M') of rank two is a 
simple vertex ofHom(P(M),Q(M')). 

Proof. Write P := P(M), Q := Q(M'). Since d = e = 2, Hom(P, Q) is a six- 
dimensional polytope and we can write 



M = { Sl ' ' ' S + m ) , M' 
ti ... t n 



To verify the conclusion of Theorem 16. II for fixed P and Q, it suffices to show the 
stronger statement that if a map / G aff (IR 2 , M 2 ) lies in the intersection of the affine 
hulls of seven facets of Hom(P, Q), then either 

• f(P) is not contained in Q, or 

• / is not of full rank. 

For each % — 1, . . . , n and j = 1, . . . , m there is a facet of Hom(P, Q) whose affine 
span is given by the equation 

(Ui,Vi) ■ f(Sj,tj) = 1. 

If we write f(x, y) = (ax + f3y + 7 , 5x + ey + () , then more explicitly, this affine 
span is given by 

fa /3 7 \ /a A 
(m Vi 0) 15 e C \[tj ]= 1, 

V° V V 1 / 
or as a linear constraint on the entries of /, 

[uiSj)a + {uitj)(5 + ua + {viSj)5 + {v{tj)e + f^C = 1- 

In other words, the condition for the affine hulls of seven facets to meet is that the 
last column in the matrix below is a linear combination of the first six columns: 

fUi 1 Sj 1 u^tj^ U{ 1 V{ 1 Sj 1 Vi 1 tj 1 Y\ 

r- 1U)7x7 



A. 



Ui 2 Sj 2 U{ 2 tj 2 Ui 2 V{ 2 Sj 2 Vi 2 tj 2 V{ 2 1 



\UijSjj U{ 7 tj 7 ttj 7 V{ 7 Sj 7 Vi 7 tj 7 V{ 7 1/ 

So the matrix Aj lv .. i j 7jlv .. j7 needs to be singular for some distinct pairs of indices 
. . . , (17, jr). Note that the indices ii, . . .i 7 need not all be distinct, nor do 

ji, ■ ■ ■ h- 

Thus it will suffice to show that for all choices of ji), . . . , (17,37), either 



HOM-POLYTOPES 



2.3 



• the seven facets do not intersect in any point of Hom(P, Q), or 

• every map in the intersection of the seven facets is of less than full rank, or 

• the generic matrix j u ...,j 7 is nonsingular. 

We index the various cases for lists of indices by coincidence graphs: bipartite 
graphs G = (A, B, E) with exactly seven edges and no isolated nodes. The set A 
represents the distinct elements of the list i\, . . .ij (facets of Q) and B represents 
the distinct elements of j\, . . . , jV (vertices of P.) The nodes of G are not labelled 
but the two parts A and B of G are distinguishable: the condition that two vertices 
of P land on the same edge line (i.e. affine span of an edge) of Q is not the same as 
the condition that a vertex of P lands on the intersection of two different edge lines 
of Q. For instance, the case that ii, . . . ,i 7 are all distinct and ji, . . . , are a ls° ah 
distinct is encoded by the graph with seven vertex-disjoint edges, the first graph in 
Table [TJ The case that ji = j 2 , 13 = £4, and everything else is distinct is encoded by 
the third graph in the left column of Table [TJ 

Observation 6.2. 

(1) If A contains a node of degree greater than two, then there is a vertex v of 
P such that f{y) is a point of intersection of three distinct edge lines of Q. 
But no three edge lines of a polygon can meet at a point (whether this point 
is inside or outside of the polygon). 

(2) If B contains a node of degree greater than two, then / sends three different 
vertices of P onto the same edge line of Q. Since any three vertices of P are 
affinely independent, this implies that / sends all of R 2 onto the same line; 
i.e, / is not of full rank. 

(3) If G contains a 4-cycle, then / sends two vertices of P to the same point: 
the intersection of two edge lines of Q. So again / is not of full rank. 

(4) If G contains a 6-cycle, then there are vertices vi,V2,vs of P and edge lines 
£1, £2, £3 of Q such that f{ Vl ) =hn e 2 , f(v 2 ) =£ 2 n £ 3 , and f{v 3 ) =£ 3 n i x . 
For the condition f(P) C Q to also be satisfied, the three intersection points 
£\ H £2, £2 n £3, and £3 n £\ must all be vertices of Q. That is, Q is a triangle. 
Then since f(P) contains all three vertices of Q, f(P) = Q, which implies 
that P is also a triangle. Then Hom(P, Q) = Q 3 , the product of three 
triangles, and then all of its vertices are simple. 

In summary, we may now assume that G has no cycles and no vertices of degree 
greater than two. That is, G is a union of vertex-disjoint paths with exactly seven 
edges. The 31 such graphs (with distinguished upper part A and lower part B) are 
shown in Table [TJ 

For each graph G, we used the computer algebra system Macaulay 2 [6J to com- 
pute the appropriate generic determinant Dq and to verify that it is not identi- 
cally zero. It follows that in each case, Dq is nonzero on a dense open subset of 
M mx2 x M 2xn , the complement of an algebraic hypersurface. Since i?2,m an d i?2n 
are full-dimensional subsets of M mx2 and IR 2xn , we conclude that Dq is also nonzero 
on a dense open subset Ug of i?2,m x R^n- We now let U n ^ m := f] G Uq, where the 
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Table 1. The 31 possible coincidence graphs 



intersection is taken over the 31 coincidence graphs G. For every pair (M, M') in 
U ntm , every full-rank vertex of Hom(P, Q) is simple. □ 

Corollary 6.3. For every m, n > 3 there is a dense open subset V m>n of R2, m x R'i n 
such that for n approaching infinity and fixed m, the ratio of simple vertices to all 
vertices in Hom(P, Q) tends to one under that condition (M, M') e V m ^ n . 

Proof. We take V mtn to be the dense open subset of U m<n given by the additional 
constraint that P has no parallel edges. For fixed (M, M') e V m ^ m we group the 
vertex maps / G Hom(P, Q) into four classes as follows. 

(1) / has rank zero; that is, / maps all of P to a single vertex of Q. Then / is 
not a simple vertex unless m — 3. 

(2) / has rank one and im(/) is an edge of Q. Again / is not simple unless 
m — 3. 

(3) / has rank one and im(/) is a proper diagonal of Q. Then by Corollary 14.3( 1). 
/ is a face collapse; that is, it maps an edge of P to one end of the diagonal 
and (since P has no parallel edges) a vertex to the other end. All other 
vertices of P are mapped into the interior of Q. This means / satisfies 
exactly six facet inequalities of Hom(P, Q), so it is simple. 

(4) / has rank two. Then / is simple by Theorem 16.11 
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By an easy refinement of Corollary 14.3( 4). we see that there are 2mn vertex maps 
of type (2), and mnin — 2) of type (3). There are also n of type (1). Let k be the 
number of vertex maps of type (4), which may depend on the particular choice of 
(M, M') e V m ,n- Regardless of this choice, the ratio of simple to total vertex maps 
is (if m > 3) at least 

mnin — 2) + k mn(n — 2) n — 2 

n + 2mn + mnin — 2) + k ~ n + mn 2 m _1 + n ' 

which tends to one as n tends to infinity. □ 

We now return to the case of arbitrary dimension, where we make the following 
conjecture which would generalize Theorem 16.11 

Conjecture 6.4. For any dimensions d and e and any integers m > d + 1 and 
n > e + 1, there is a dense open subset Ud,e,m,n of Rd, m x R' e „ such that if (M, M') 6 
Ud, e ,m,n, then every vertex map f : P(M) — » Q(M') of full rank is a simple vertex 
o/Hom(P,Q). 

The motivation is that the main idea of the proof of Theorem 16.14 which is 
to use a finite list of graphs to enumerate the situations under which a vertex of 
Hom(P(M), Q(M) might not be simple and then to show that each is equivalent to 
the nonvanishing of a certain generic determinant, does not appear to depend on the 
dimensions of the source and target polytopes. However, some of the conditions in 
Observation 16.21 do not apply in the more general setting, and the number of graphs 
that must be checked grows very rapidly with d and e. 



7. Regular polygons 

We now consider the hom-polytopes of pairs (P m , P n ) of regular polygons. There 
do not appear to be general descriptions of all vertices of Hom(P m ,P n ), or even of 
Hom(P n , P n ), so we only consider special cases. Again the main goal is to understand 
the set of vertices. In fact we need only consider those of full rank, since we have 
the following immediate consequence of Corollary 14.3( 4). 

Corollary 7.1. The number of rank zero vertices of Hom(P m , P n ) is n and the 

number of rank one vertices is 

mnin — 1) for m odd, 
— ^ — '- for m even. 

For any positive integer k > 3, let be the cyclic group of order k and let Dj. 
be the dihedral group of order 2k. Then Dk acts naturally on P^ by rotation and 
reflection, with identified with the subgroup of rotations inside Dk- In particular, 
this induces an action of D rn x D n on Hom(P m ,P n ). The action respects faces of 
each dimension. The action of the subgroup Z m x Z n is free and transitive on the 
mn facets of Hom(P m , P n ). Although the action on the vertices is more subtle, it is 
still useful in enumerating them. We begin with a sample result that holds for all 
m and n and serves as a mild check on the experimental data we will later present. 
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Proposition 7.2. Fix m and n and let V be the number of vertices o/Hom(P m , P n ). 



(1) n divides V, 

(2) m divides V — n, and 

(3) n and V have the same parity. 

Proof. All of these statements follow from applying the orbit-stabilizer theorem to 
the action of various subgroups of D m x D n on vert(Hom(P m , P„)). 

(1) The action of any nontrivial rotation in Z n on Hom(P m , P n ) fixes only one affine 
map: the zero map. The zero map is not a vertex of Hom(P m , P n ), so the action is 
free on the vertices. 

(2) The action of any nontrivial rotation in Z m on Hom(P m ,P n ) fixes only vertex 
maps of rank zero. Thus m divides V — n. 

(3) If n is even, this is immediate from (1). 

If n is odd, then a reflection s G D n fixes only the line segment from one vertex 
of P n to the midpoint of the opposite edge. Thus the only maps in Hom(P m , P„) 
that are fixed by s are the maps that take P m to that segment. But then, by the 
perturbation criterion (pc 1 ), the only vertex of Hom(P m ,P n )) that is fixed by s is 
the map that takes all of P m to the mentioned vertex of P n . It follows that 2 divides 
V - 1, so V is odd. □ 

Proposition 7.3. For any n>3, Hom(P 3 ,P„) = P%. In particular, the number of 
full-rank vertices is n(n — l)(n — 2). 

Proof. The first statement is a special case of Corollary 13.6( 1). For the second, note 
that a map / is a full rank vertex of Hom(P3, P n ) if and only if it sends each vertex 
of P3 to a distinct vertex of P n . □ 

Proposition 7.4. The number of full rank vertices of Hom(P ra , P3) is 



Proof. Suppose / is a rank two vertex of Hom(P, P3) for any polygon P. Then / 
cannot send more than two vertices to any edge of P3. Since it lies at the intersection 
of at least six different facets of Hom(P, P 3 ), it must send exactly two vertices of P 
to each edge of P3. That is, it inscribes an affine image of P into the equilateral 
P3. But since all triangles in the plane are affinely isomorphic, this is equivalent to 
circumscribing an arbitrary triangle around P itself. 

Now specialize to P = P n and let its edges be E ,Ei,... , P„_i in consecutive 
order. The condition that a triangle can be drawn around P n along three edges 
E iy Ej,Ek is that the gap between each pair of indices, taken cyclically, is at most 
[(n — l)/2j . We count such triples separately in the cases where n is odd and where 
it is even. 

Suppose n = 2p + 1 is odd. Consider just the triples = i < j < k. Then we 
must have 



Then 




for n odd 
for n even 



1 < j < Pi k — j < p, {2p + 1) — k < p. 
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That is, for j fixed we must have p + l<k<p + j, giving Y^=i3 = ( P 2 ) triples 
with i — 0. Since - of the allowed triples include 0, the total number of triples is 

nfp+\\ (p + l)p(2p+l) 

6 

(2p + 2)2p(2p + 1) 



24 

(n + l)(n — l)n 
24 ' 

Similarly if n = 2p is even, the size of each gap must be at most p — 1. A similar 
calculation gives the formula n ( n ~^K"~ 4 ) f or the number of triples of edges in this 
case. 

(Note that these formulae appear without proof as sequence A060422 in [11] , listed 
as the number of triples of vertices of a regular ra-gon that form acute triangles. By 
passing to the dual polygons with respect to 0, it is not hard to see that this is 
equivalent.) 

Finally, once we have chosen the three edges Ei,Ej,Ek of P n , we can apply an 
arbitrary symmetry of P3, so we multiply the formulae above by 3! = 6 to obtain 
the total number of rank two vertices of Hom(P n , P3). □ 

Proposition 7.5. The number of vertices of Hom(P m , P4) is 

(2m + 2) 2 for m odd 
(m + 2) 2 for m even. 

In particular the number of full-rank vertices is 

4m 2 — 4m for m odd 
m 2 — 2m for m even 

Proof. By Proposition 0[3), Hom(P m , P 4 ) = Hom(P m ,J) x Hom(P m ,J). Now 
Hom(P m , J) has two vertices of rank zero and (by Corollary 14. 3 j) has 2m vertices of 
rank one if m is odd or m such vertices if m is even. □ 

Note that if m is even, we obtain an explicit description of Hom(P m , 7) from 
Corollary 13.6( 2): it is a bipyramid over the dual m-gon P^. The /-vector (i. e., 
the vector of the numbers of vertices, edges, 2-faces etc) of this bipyramid is (m + 
2, 3m, 2m, 1). Using the general formula 

i 

f i (PxQ) = Y,fj(P)fi-AQ), 

j=0 

we compute the /-vector of Hom(P m , J) to be 

(m 2 + 4m + 4, 6m 2 + 12m, 13m 2 + 4m, 12m 2 + 2m + 4, 4m 2 + 6m, 4m, 1). 

Proposition 7.6. The number of full-rank vertices o/Hom(P 4 ,P n ) is 

n 3 — 9n for n odd 

n 3 — 5n 2 + 6n for n even 
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Proof. Let vq, t>i, t> 2 , v 3 and w , u>i, . . . , w n _i respectively be the vertices of P 4 and of 
P n in counterclockwise order, with Vq = Wq. If / : P4 — > P n is a vertex map of rank 
two, then in order to achieve the necessary six incidences of vertices of /(-P4) with 
facets of P n , one of the following must apply: 

(1) / sends at least three vertices of P 4 to distinct vertices of P n , or 

(2) / sends two adjacent vertices of P 4 to distinct vertices of P n and the other 
two onto interior points of edges, or 

(3) / sends two opposite vertices of P4 to distinct vertices of P n and the other 
two onto interior points of edges. 

In case (1), we first observe that an affine map / : P4 — > P n which maps three 
vertices of P4 to vertices of P n is automatically a vertex map; see the comment at 
the beginning of Example 15.11 We may assume up to symmetry that J{vq) = wq, 
f(vi) = Wi, and f(v 3 ) = u>„-j with < i < j. We now consider different cases for 
the interior angle G of the parallelogram /(P4) at Wo- 
lf 9 is acute, then f(v 2 ) lies outside the unit circle. In particular it is not in P n , 
so no vertex maps are obtained this way. 

If is a right angle, then /(i>2) is a rectangle and we must have that n = 2p is 
even and i + j = p. Furthermore f(v 2 ) is also a vertex of P n : specifically it is the 
vertex w p directly opposite Wq, as in the first two pictures in Table |3j Thus to fix 
the image of such a map, we must pick two opposite pairs of vertices of P n . Finally, 
taking into account the eight possible orientations of /(P4), the number of these 
maps is 



Finally, if is obtuse then f(v 2 ) is necessarily in the interior of P n (one uses the 
symmetry w.r.t. to the perpendicular line through the midpoint of [%w n _j]), In 
particular, / is a valid map from P 4 to P n . The condition for to be obtuse is that 
% + j < n/2. This situation holds in the third and fourth pictures in Table [2] (with 
n = 7 and respectively i = j = 1; i = l,j = 2) and in the third and fourth pictures 
in Table |3] (with n = 8 and respectively i=j = l;i = l,j = 2.) 

By considerations similar to the above, the number of ways to obtain such a map 

is 



In case (2), we may assume that f(v ) = w and /(t>i) = u>j for some i < n/2. 
Then, since P n is symmetric w.r.t. the perpendicular line through the midpoint 
of [wo,w«], we must actually have that /(P4) is a rectangle in order that both t> 2 
and t>3 land on edges. Furthermore n must be odd or f{v 2 ) and f{v%) will both be 
vertices of P n , a situation we already considered in case (1). Finally, we must have 





if n = 2p + 1, or 




— 6n 2 + 8n 



if n = 2p. 
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Table 3. Rank two vertices of Hom(P4, Pg) up to symmetry 



i + 1 < n/2 in order that the right angles at f(vo) and f(v\) are contained in P n . 
This situation holds in the first and second pictures in Table |2l with n = 7 and 
respectively i — 1; i — 2. 

Setting n = 2p + 1, we see that the number of maps of this type is 

8(2p + l)(p - 1) = 4n(ra - 3) = 4n 2 - 12n. 

That all these maps are in fact vertex maps follows from (pc 1 ): any affine 1-family 
must be constant on t> and V\, but if such a family is not constant on t> 2 then 
the images of V2 and v%, when the family parameter varies over (—1,1), must trace 
out parallel lines - something not possible because these images are confined to 
non-parallel edges of P n . 

In case (3), we may assume f(v ) = wo and f{v2) = Wi for some < i < n/2. 
If i = n/2, then n is even. Then we can indeed arrange that f(vi) and /(v^) are 
both on edges, but these will necessarily be opposite (and hence parallel) edges 
of P n . Without moving vq or t> 2 , we can then slide vi and along these edges 
in the opposite directions to produce an affine 1-family of maps, contradicting the 
assumption that / is a vertex map. 

On the other hand, if % < n/2, and we assume that f{v\) is on an edge, let 
Q be the polygon with vertices wo, w±, . . . ,Wi and Q' be the polygon obtained by 
reflecting Q across the line defined by wo and Since f(vo) = wq, f{v2) = Wi, and 
f(wi) G dQ, we conclude that f(ws) G dQ'. But Q' touches the edges of P only at 
Wq and at Wi, so we cannot arrange that f(v 3 ) lies on an edge of Q. That is, there 
are no vertices of Hom(P 4 , P m ) of this type: case (3) is impossible. 

Adding up the vertices described by the various cases, we obtain the total count 
of rank two vertices of Hom(P 4 , P n ) as claimed in the proposition. □ 
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Table 4. Expected numbers of vertices of Hom(P m , P n ) 



7.1. Experimental results. We end with a table of experimental results for the 
number of vertices of Hom(P m , P n ) for all m,n < 8. Our approach to the com- 
putation was as follows. We begin with rational approximations Q m and Q n and 
compute the polytope H : = Hom(Q m , Q n ) with exact arithmetic, using the software 
package Polymake [5J. This is possible because of the explicit facet description of 
any hom-polytope given by Proposition 12.11 

However, we do not expect Rom(Q m , Q n ) to have the same number of vertices as 
Hom(P m , P n ). For example, if m = n > 5, the affine map that rotates P m by — is a 
vertex of Hom(P m , P m ) at which 2m facets meet; see Example 15.41 This map does 
not exist in Hom(Q m , Q m ). Specifically, the corresponding facets do not all meet 
in one point, but various subsets of them do meet to form several different vertices 
of Hom(Q m , Qm). Our problem, then, is to identify the collections of vertices of 
Hom(Q m , Q n ) that correspond to single vertices of Hom(P m , P n ). 

Given a polytope R and e > 0, we say that a collection V of vertices of R is an 
e-cluster if \\v — w\\ < e for all v,w £ V. If all of the vertices of R are partitioned 
into a collection of disjoint clusters, we say that e successfully partitions vert(-R). 

Note that for any sufficiently large e, we get a single cluster, and for any sufficiently 
small e, each vertex forms a cluster by itself. However, we need intermediate values 
of e that successfully partition vert(Hom(Q m , Q n )). By trying several values, we find 
that for six-digit rational approximations Q m ,Qn and for all m,n < 8, the values 
e = 10 -3 and e = 10~ 4 give the same nontrivial partition. Furthermore, the data 
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resulting from such a partition agree with what we have proved for the cases m — 3, 
m = 4, n = 3, and n = 4, and also with Proposition 17.21 

Using this partition, we predict the vertex counts shown in the slanted entries of 
Table HI all other values can be obtained from the theoretical results in this section. 
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